Computer system for performing remote copy using journal

ABSTRACT

An object of the invention is to maintain conformability of data stored in a secondary storage system although a time stamp is not granted to write data received by a primary storage system. At least one of one or more computers includes a storage managing unit. The storage managing unit issues an ID next to an immediately previously issued ID to one or more primary storage systems (PDKCs) at regular or irregular intervals. Upon receiving a write command from one of the one or more computers, each PDKC prepares a journal (JNL) including a replica (journal data) of write data and the newest ID from the storage managing unit, and transmits the journal to a secondary storage system (SDKC). The SDKC stores a received JNL and reflects up to a particular JNL of non-reflected JNLs in a secondary volume. The particular JNL has an ID older by one than the oldest ID of the newest arrival completion IDs in one or more SDKCs.

CROSS REFERENCES TO RELATED APPLICATIONS

This application relates to and claims priority from Japanese PatentApplication No. 2008-272590, filed on Oct. 23, 2008, the entiredisclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to remote copy using a journal.

2. Description of the Related Art

For example, Patent Document 1 discloses a technique in which a primarystorage system receives write data from a primary host computer andreports to the primary host computer that the receipt of write data hasbeen completed immediately after the write data was received.Thereafter, the host computer reads a copy of the write data from theprimary storage system. The write data is granted with a write timewhich is a time at which a write request for the write data is issued,and, when the write data is read by the primary host computer, the writetime is passed to the primary host computer. In addition, the primaryhost computer sends the write data and the write time to a secondaryhost computer. Upon receiving the write data and the write time, thesecondary host computer writes information such as the write time in acontrol volume of a secondary storage system and further writes thewrite data in the secondary storage system in a sequence of write timeby referring to the write time granted to the write data.

[Patent Document 1] Europe Patent Application Laid-Open No. 0672985

SUMMARY OF THE INVENTION

In the technique disclosed in Patent Document 1, data consistency ismaintained by using the write time (time stamp) granted to the writedata. On this account, this technique cannot be applied to a computersystem that does not grant the write time to the write data.Specifically, for example, although the technique disclosed in PatentDocument 1 may be applied to a computer system having a so-calledcomputer which can grant the write time for a write request, thistechnique cannot be applied to a computer system having a so-called opensystem computer which does not grant the write time for the writerequest.

It is therefore an object of the invention to maintain conformability ofdata stored in a secondary storage system although a time stamp is notgranted to write data received by a primary storage system.

At least one of one or more computers includes a storage managing unit.The storage managing unit issues an ID next to an immediately previouslyissued ID to one or more (for example, two or more) primary storagesystems at regular or irregular intervals. Upon receiving a writecommand from one of the one or more computers, each primary storagesystem writes write data in a primary volume, prepares a journalincluding journal data, which is a replica of the write data, and thenewest ID from the first computer, stores the prepared journal in aprimary journal memory region, and transmits the journal to a secondarystorage system of a connection site. The secondary storage system writesa received journal in a secondary journal memory region, and writesjournal data in journal up to a particular journal of one or morejournals stored in the secondary journal memory region in a secondaryvolume. The particular journal is a journal having an ID older by onethan the oldest ID of the newest arrival completion IDs in the one ormore secondary storage systems.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of configuration of a computer system accordingto an embodiment of the present invention.

FIG. 2 is an explanatory view of exchange between PDKC and SDKC.

FIG. 3 is a view showing an outline of setting of restore permission TS#and restore process.

FIG. 4 is a view showing a configuration of each DKC.

FIG. 5 is a view showing a functional block of PDKC and a functionalblock of SDKC.

FIG. 6A is a view showing an example of configuration of a pairmanagement table 57P1.

FIG. 6B is a view showing an example of configuration of a pairmanagement table 57S1.

FIG. 7 is a view showing a relation between a JNL group and aconsistency group.

FIG. 8 is a view showing a configuration of a JNL group managementtable.

FIG. 9 is a flow chart showing a process performed by PDKC uponreceiving a WR command.

FIG. 10 is a flow chart showing a process performed by SDKC upontransmitting an RD-JNL command.

FIG. 11 is a part of a flow chart showing a process performed by PDKCupon receiving an RD-JNL command.

FIG. 12 is the remaining of a flow chart showing a process performed byPDKC upon receiving an RD-JNL command.

FIG. 13 is a flow chart showing a process performed by SDKC uponreceiving RD data.

FIG. 14 is a view showing an example of a definition file used by RM.

FIG. 15 is a view showing information elements included in a TS settingcommand.

FIG. 16 is a flow chart showing a process performed by PDKC uponreceiving a TS setting command from RM.

FIG. 17 is a flow chart showing a process to determine restorepermission TS#.

FIG. 18 is a view showing examples of information elements included inJNCB.

FIG. 19 is a view showing examples of information elements included inan RD-JNL command parameter.

FIG. 20 is a view showing examples of information elements included inresponse information to a state acquisition command from RM.

FIG. 21 is a flow chart showing a process performed by PDKC uponreceiving a state acquisition command from RM.

FIG. 22 is an explanatory view of the reason why data up to JNL, whichis less by one than restore permission TS#, is reflected.

FIG. 23A is an explanatory view of a problem which may occur in theabsence of a function to transmit NO-JNL.

FIG. 23B is an explanatory view of an effect which may be expected inthe presence of a function to NO-JNL.

FIG. 24 is an explanatory view of any process in a case where a freezecommand is transmitted before a TS setting command is transmitted.

FIG. 25 is an explanatory view of a problem which may occur in a casewhere a freeze command is not transmitted before a TS setting command istransmitted.

FIG. 26 is a flow chart showing a process performed by PDKC uponreceiving a freeze command.

FIG. 27 is a flow chart showing a process performed by PDKC uponreceiving a RUN command.

FIG. 28 is a view showing occurrence of disturbance in RM.

FIG. 29 is a flow chart showing a process performed when RM is restoredout of a disturbance.

FIG. 30 is a view showing a process performed when there occurs adisturbance between RM and PDKC 11 P1.

FIG. 31 is a view showing a process performed when the disturbance shownin FIG. 30 is removed.

FIG. 32 is a continuation of the process shown in FIG. 31.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, an embodiment of the present invention will be describedwith reference to the drawings. In the following description, a storagesystem is symbolized by “DKC,” a primary DKC is symbolized by “PDKC,”and a secondary DKC is symbolized by “SDKC.” In addition, the terms“write” and “read” may be sometimes abbreviated as “WR” and “RD,”respectively.

To begin with, an outline of this embodiment will be described withreference to FIGS. 1 to 3.

FIG. 1 shows an example of configuration of a computer system accordingto an embodiment of the present invention.

There is provided a DKC group 18, and the DKC group 18 has the form ofan M×N matrix. That is, the DKC group 18 includes M PDKCs belonging to aprimary site (P site) 15P and N SDKCs belonging to a secondary site (Ssite) 15S. M is an integer larger than one and N is also an integerlarger than one. At least one SDKC is connected to a PDKC and at leastone PDKC is connected to a SDKC. Accordingly, for example, one SDKC maybe connected to two PDKCs. In this embodiment, the DKC group 18 includestwo PDKCs (M=2) 11P1 and 11P2, and two SDKCs (N=2) 11S1 and 11S2. ThePDKCs and the SDKCs are interconnected in a one-to-one correspondence.That is, the PDKC 11P1 is connected to and communicates with the SDKC11S1 and the PDKC 11P2 is connected to and communicates with the SDKC11S2.

The PDKC 11P1 includes a PVOL 17P1 and a JVOL 17PJ1 and the PDKC 11P2includes a PVOL 17P2 and a JVOL 17PJ2. The SDKC 11S1 includes a JVOL17SJ1 and a SVOL 17S1 and the SDKC 11S2 includes a JVOL 17SJ2 and a SVOL17S2.

Here, “PVOL,” which is an abbreviation of a primary volume, refers to alogical volume designated by a write command from a host. Write data arewritten in the PVOL according to the write command. “JVOL,” which is anabbreviation of a journal volume, refers to a logical volume in which ajournal (JNL) is written. “JNL” is information indicating an updatehistory of PVOL, for example, information including JNCB and JNL data.“JNCB” is management information related to JNL (JNL managementinformation). The JNCB includes, for example, a sequence number (SEQ#)indicating an order at which JNL having the JNCB is prepared, updateposition information indicating a position (for example, a block) of aPVOL in which write data corresponding to JNL data in the JNL arewritten, a P site receipt TS# which will be described later, etc. “JNLdata” is a replica of write data.

“SVOL,” which is an abbreviation of a secondary volume, refers to alogical volume paired with the above-mentioned PVOL. In the SVOL arewritten the JNL data which are a replica of the write data stored in thepaired PVOL (that is, data stored in the PVOL are also stored in theSVOL).

Any logical volumes of the above-mentioned PVOL, JVOL and SVOL arelogical memory devices formed on the basis of a plurality of HDDs (HardDisk Drives) (or other physical memory devices such as flash memories)of the DKC.

The computer system further includes a plurality of host computers(hereinafter abbreviated as “host”), for example, two hosts 21A and 21B.The hosts 21A and 21B include their respective I/O issuing units 25.Each I/O issuing unit 25 is implemented by an application program and/oran operating system (OS). The OS is an OS of an open system. The I/Oissuing unit 25 transmits an I/O command, i.e., a write command (WRcommand) or a read command (RD command). In the I/O command is specifieda PVOL of an I/O destination (specifically, for example, a set of a portnumber and a LUN (Logical Unit Number)).

Only one 21A of the hosts 21A and 21B is equipped with a RAID manager(RM) 23 and at least the host 21A is connected to both of the PDKCs 11P1and 11P2. The RM 23 is a computer program executed by a microprocessorand issues a time stamp number (TS#) to both of the PDKCs 11P1 and 11P2regularly. The TS# is a simple number, not time. The TS# may be eitherissued to each of the PDKCs in order or transmitted to both of the PDKCs11P1 and 11P2 simultaneously. The same first TS# may be transmitted toboth of the PDKCs 11P1 and 11P2 at a first time. At the next time, a TS#next to the first TS# (a number of (the TS# at the first time)+1) istransmitted to both of the PDKCs 11P1 and 11P2. Whenever receiving theTS#, each of the PDKCs 11P1 and 11P2 updates the P site receipt TS# tothe received TS#. As a result, in each of the PDKCs 11P1 and 11P2, thenewest TS# of the received TS# is managed as the P site receipt TS#.

Hereinafter, a process performed by the PDKC 11P1 and the SDKC 11S1 willbe described with a set of the PDKC 11P1 and SDKC 11S1 as a mainexample.

Upon receiving the WR command, the PDKC 11P1 performs the following twooperations (1-1) and (1-2): (1-1) writing the WR data according to theWR command in the PVOL 17P1 specified by the WR command; and (1-2)preparing JNL including JNL data as a replica of the WR data and writingthe JNL in the JVOL 17PJ1. A JNCB in the JNL prepared in the (1-2)operation includes a SEQ# of the JNL, update position information (forexample, an ID of PVOL of a WR destination of the WR data (for example,a number or LUN of the PVOL) or an address of a WR destination block inthe PVOL) or the P site receipt TS# managed by the PDKC 11P1 whichreceived the WR command.

The PDKC 11P1 transmits JNL, which has not yet transmitted to the SDKC11S1, of JNLs stored in the JVOL 17PJ1, to the SDKC 11S1.

Upon receiving the JNL transmitted from the PDKC 11P1, the SDKC 11S1writes the received JNL in the JVOL 17SJ1.

In addition, the SDKC 11S1 reflects JNL, which has not yet reflected inthe SVOL 17S1, of the JNLs stored in the JVOL 17SJ1, in the SVOL 17S1.The phrase “reflects JNL” means that JNL data in the JNL are written inthe SVOL paired with the PVOL specified by the update positioninformation of the JNCB in the JNL, that is, the data are restored. Atthat time, a WR destination of the JNL data becomes a block (in theSVOL) of the same address as or an address corresponding to a WRdestination block address specified by the update position information.The reflection of the JNL is carried out in a SEQ# order of the JNL.

As shown in FIG. 2, the SDKC 11S1 manages a restore permission TS#, anarrival completion TS# and a restore completion TS#. “Restore permissionTS#” refers to a TS# used to specify how far JNL can be reflected (therestore permission TS# will be described in detail later). “Arrivalcompletion TS#” refers to TS# of the newest JNL (JNL having the newestSEQ#) of received JNLs. “Restore completion TS#” refers to TS# of thenewest JNL (JNL having the newest SEQ#) of JNLs reflected in the SVOL17S1.

As shown in FIG. 2, the PDKC 11P1 transmits a JNL in response to anRD-JNL command from the SDKC 11S1. Specifically, for example, the SDKC11S1 transmits the RD-JNL command to the PDKC 11P1 at regular orirregular intervals. “RD-JNL command” refers to an RD command of JNL.The SDKC 11S1 includes the arrival completion TS# and the restorecompletion TS#, which are managed by the SDKC 11S1, in the RD-JNLcommand. Upon receiving the RD-JNL command, the PDKC 11P1 performs thefollowing two operations (2A-1) and (2A-2): (2A-1) storing the arrivalcompletion TS# and the restore completion TS# in the RD-JNL command; and(2A-2) transmitting a JNL, which has not yet transmitted to the SDKC11S1, of the JNLs stored in the JVOL 17PJ1, to the SDKC 11S1, as RD datafor the RD-JNL command. In the RD-JNL command may be specified a SEQ# ofa JNL next to the newest JNL of the JNLs received by the SDKC 11S1 (thenewest JNL of the JNLs stored in the JVOL 17SJ1). In this case, the JNLhaving the SEQ# may be read from the JVOL 17PJ1, transmitted from thePDKC 11P1 to the SDKC 11S1, and written in the JVOL 17SJ1.

Upon receiving a JNL as RD data from the PDKC 11P1, the SDKC 11S1performs the following two operations (2B-1) and (2B-2): (2B-1) storinga TS# in the received JNL as the arrival completion TS#; and (2B-2)writing the received JNL in the JVOL 17SJ1. Additionally, as shown inFIG. 2, a P site arrival TS# is not managed or managed as a null value“0” in the SDKC 11S1.

In addition, when a non-reflected JNL in the JVOL 17SJ1 is reflected inthe SVOL 17S1, the SDKC 11S1 stores a TS# in the reflected JNL as therestore completion TS#.

As the arrival completion TS# and the restore completion TS# managed bythe SDKC 11S1 (11S2) are included in the RD-JNL command of a destinationof the PDKC 11P1 (11P2), the arrival completion TS# and the restorecompletion TS# are transmitted to and stored in the PDKC 11P1 (11P2). Asshown in FIG. 3, the arrival completion TS# stored in the PDKC 11P1(11P2) is transmitted from the PDKC 11P1 (11P2) to the RM 23.

That is, as shown in FIG. 3, the RM 23 first transmits a freeze commandto the PDKCs 11P1 and 11P2 (S301A and S301B). “Freeze command” refers toa command to reserve an I/O process. The PDKCs 11P1 and 11P2 receivedthe freeze command enter into a freeze state (S302A and S302B) duringwhich, although the PDKCs 11P1 and 11P2 receive an I/O command from thehost 21A or 21B, the PDKCs 11P1 and 11P2 do not perform an I/O operationfor the PVOL specified by the I/O command (for example, the PDKCs 11P1and 11P2 do not respond to the host 21A or 21B although the PDKCs 11P1and 11P2 receive the I/O command from the host 21A or 21B).

Next, the RM 23 transmits a report command (a state acquisition commandto be described later) to the PDKCs 11P1 and 11P2 (S303A and S303B).Upon receiving the report command, the PDKC 11P1 (and 11P2) transmit anarrival completion TS# 203 (arrival completion TS# 201) managed by thePDKC 11P1 (11P2) to the RM 23 (S304A (S304B)).

The RM 23 determines the lowest value 201 of the two arrival completionTS#s received from the PDKCs 11P1 and 11P2, as a restore permission TS#201 (S305).

The RM 23 transmits the restore permission TS# 201 to the PDKCs 11P1 and11P2 (S306A and S306B). The PDKC 11P1 (11P2) stores the restorepermission TS# 201 received from the RM 23 and transmits the storedrestore permission TS# to the SDKC 11S1 (11S2) (S307A (S307B)) The SDKC11S1 (11S2) stores the received restore permission TS# 201.

The SDKC 11S1 (11S2) reflects up to a JNL having a TS#, which is smallerby one than the restore permission TS# 201 (that is, a TS# 200), ofnon-reflected JNLs stored in the JVOL 17SJ1 (17SJ2), in the SVOL 17S1(17S2) in a SEQ# order (S308A (S308B)

Hitherto, the outline of this embodiment has been described. In theabove description, at least one of the following items (3-1) to (3-4)may be employed.

(3-1) At least one of the arrival completion TS# and the restorecompletion TS# may be communicated from the SDKC 11S1 (11S2) to the RM23. However, as described previously, when the arrival completion TS#and the restore completion TS# are communicated to the RM.23 via thePDKC 11P1 (11P2), the host 21A having the RM23 may not be connected tothe SDKC 11S1 (11S2) (or its connection may not be changed).

(3-2) The restore permission TS# itself may be a TS# smaller by one thanthe lowest value of the arrival completion TS#. In this case, at S308Aand S308B, up to a JNL having the restore permission TS# is reflected.

(3-3) The RM 23 may be equipped in a management computer for managingthe PDKCs 11P1 and 11P2, which is not a host.

(3-4) A JNCB in a JNL may be managed as a memory region (for example, amemory in a controller of the DKC) other than the JVOL.

As can be seen from the above description, it is possible to maintainconformability of data stored in one or more SDKCs although a hostcannot grant a time stamp to a WR command.

Hereinafter, this embodiment will be described in detail.

FIG. 4 shows a configuration of each DKC.

As described above, at least the host 21A is connected to the PDKCs 11P1and 11P2, and the PDKC 11P1 (11P2) and the SDKC 11S1 (11S2) communicatewith each other.

Although the DKCs 11P1, 11P2, 11S1 and 11S2 may have the sameconfiguration or different configurations, it is assumed in thisembodiment that they have the same configuration. Specifically, forexample, the PDKC 11P1 (11P2, 11S1 and 11S2) has a plurality of physicalmemory devices as basic logical volumes and a controller for controllingan I/O for the plurality of physical memory devices. The controllerincludes, for example, a plurality of ports 31P1 (31P2, 31S1 and 31S2),a plurality of CHAs 32P1 (32P2, 32S1 and 32S2), a plurality of CMs 33P1(33P2, 33S1 and 33S2), a plurality of SMs 34P1 (34P2, 34S1 and 34S2) anda plurality of DKAs 35P1 (35P2, 35S1 and 35S2).

The port 31P1 (31P2) receives a TS# from the host 21A or an I/O commandfrom the host 21A and/or 21B. The port 31S1 (31S2) receives RD data(JNL) from the PDKC 11P1 (11P2).

CHA is an abbreviation of a channel adapter. The CHA 32P1 (32P2) is aninterface device which includes the port 31P1 (31P2), a microprocessor,a memory and so on and controls communication with the hosts 21A and21B. At least one of the plurality of CHAs 32P1 (32P2) is connected tothe port 31S1 (31S2) of the CHA 32S1 (32S2) and controls communicationwith the CHA 32S1 (32S2) (communication with the SDKC 11S1 (11S2)).

CM is an abbreviation of a cache memory. The CM 33P1 (33P2) is a memorywhich temporarily stores data (WR data and RD data) exchanged betweenthe PVOL 17P1 (17P2) and the hosts 21A and 21B. The CM 33S1 (33S2) is amemory which temporarily stores data to be written in or read from theSVOL 17S1 (17S2).

SM is an abbreviation of a share memory. The SM 34P1 (34P2, 34S1 and34S2) is a memory accessed by the CHA 32P1 (32P2, 32S1 and 32S2) and theDKA 35P1 (35P2, 35S1 and 35S2). The SM 34P1 (34P2, 34S1 and 34S2) storea variety of management information. The management information mayinclude, for example, configuration information of the DKC 11P1 (11P2,11S1 and 11S2) (for example, information indicating which VOL is formedon the basis of which physical memory device, an ID of the VOL, etc.), apair management table and a JNL group management table (which will bedescribed later).

DKA is an abbreviation of a disk adapter. The DKA 35P1 (35P2, 35S1 and35S2) is an interface device which includes a microprocessor, a memory,etc. and controls communication with physical memory devices.

Hereinafter, the function of the PDKC 11P1 and SDKC 11S1 will bedescribed by the examples of PDKC 11P1 and SDKC 11S1.

FIG. 5 shows a functional block of the PDKC 11P1 and a functional blockof the SDKC 11S1.

At least one of the plurality of ports 31P1 is connected to the host 21Aand at least one of the plurality of ports 311 is connected, as aninter-DKC port (initiator port), to the port 31S1 as an inter-DKC port(target port) via, for example, a cable for fibre channel. The RD-JNLcommand and the RD data are transmitted/received via this inter-DKCport.

In the PDKC 11P1, for example, the CHA 32P1 includes a TS processingunit 51P1, an I/O processing unit 52P1, a JNL preparing unit 53P1, aJNL-RD receipt processing unit 54P1, a F/R receipt processing unit 55P1and a pair state modifying unit 56P1 and the SM 34P1 stores a pairmanagement table 57P1 and a JNL group management table 58P1. On theother hand, in the SDKC 11S1, for example, the CHA 32S1 includes aJNL-RD processing unit 61S1, a restore processing unit 62S1 and a pairstate modifying unit 56S1 and the SM 34S1 stores a pair management table57S1 and a JNL group management table 58S1. Each of the above components51P1 to 56P1, 61S1, 62S1 and 56S1 is, for example, a computer programexecuted by a microprocessor, but all or some of at least one of thecomponents may be configured as hardware.

The TS processing unit 51P1 processes a TS# from the RM 23.

The I/O processing unit 52P1 processes an I/O according to an I/Ocommand from a host.

The JNL preparing unit 53P1 prepares a JNL including JNL data, which area replica of WR data written in the PVOL 17P1, etc., and writes theprepared JNL in the JVOL 17PJ1.

The JNL-RD receipt processing unit 54P1 reads the JNL from the JVOL17PJ1 in response to an RD-JNL command.

The F/R receipt processing unit 55P1 performs a process in response to afreeze command and a RUN command from the RM 23. The term “RUN command”used herein refers to a command to release a freeze state.

The pair state modifying unit 56P1 (56S1) modifies a pair state of thePVOL 17P1 (17P2) and the SVOL 17S1 (17S2).

The JNL-RD processing unit 61S1 transmits the RD-JNL command.

The restore processing unit 62S1 reflects a non-reflected JNL stored inthe JVOL 17SJ1 (17SJ2) in the SVOL 17S1 (17S2).

The pair management table 57P1 (57S1) is a table for managing a pairconfiguration of PVOL and SVOL and a pair state thereof.

The JNL group management table 58P1 (58S1) is a table for managinginformation related to a JNL group.

FIG. 6A shows an example of configuration of the pair management table57P1. FIG. 6B shows an example of configuration of the pair managementtable 57S1.

With the pair management table 57P1 (57S1) are registered a volumenumber, a volume attribute, a JNL group number, a pair state, acounterpart volume number and a counterpart JNL group number for eachPVOL (SVOL). Hereinafter, details of one VOL will be described by way ofan example of “object volume” with reference to FIGS. 6A and 6B.

The volume number refers to an identification number of an objectvolume. The volume number is associated with, for example, a LUN.Accordingly, for example, from a LUN specified by a WR command, a volumenumber associated with the LUN is specified, and a PVOL of a WRdestination is specified from the specified volume number.

The volume attribute refers to an attribute of an object volume. A valuerepresenting the attribute may include, for example, “P” representing aPVOL and “S” representing a SVOL.

The JNL group number refers to an identification number of a JNL groupto which an object volume belongs.

The pair state refers to a pair state of an object volume. Examples ofthe pair state may include “PAIR” meaning that a PVOL has the samecontents as a SVOL, “Suspend” meaning that a copy of a PVOL to a SVOL issuspended, and “Copy” meaning that a copy of a PVOL to a SVOL isperformed.

The counterpart volume number refers to an identification number of avolume paired with an object volume.

The counterpart JNL group number refers to an identification number of aJNL group paired with a JNL group to which an object volume belongs.

From the table 57P1 shown in FIG. 6A, it can be seen that, in the PDKC11P1, a PVOL having a volume number “0000” is paired with a SVOL havinga volume number “1000.” On the other hand, from the table 57S1 shown inFIG. 6B, it can be seen that, in the SDKC 11S1, a SVOL having a volumenumber “1000” is paired with a PVOL having a volume number “0000.”

Here, the JNL group will be described with reference to FIG. 7. The JNLgroup refers to a set of a plurality of VOLs in each DKC. The JNL groupincludes one or more JVOLs and one or more PVOLs (SVOLs). A JNL group inthe PDKC is paired with a JNL group in the SDKC. A consistency group isformed by a plurality of JNL group pairs. In this embodiment, dataconformability is maintained in this consistency group.

FIG. 8 shows an example of configuration of the JNL group managementtable 58P1 (58S1).

The JNL group management table 58P1 (58S1) is a table prepared for eachJNL group. Information elements included in the JNL group managementtable 58P1 (58S1) may include, for example, a JNL group state, a JVOLvolume number, a JNL group number, various SEQ#s, an OPEN M*N validflag, freeze information, freeze state setting time, a P site receiptTS#, a restore permission TS#, an arrival completion TS# and a restorecompletion TS#. Accordingly, the P site receipt TS#, the restorepermission TS#, the arrival completion TS# and the restore completionTS# are managed for each JNL group in one DKC.

The JNL group number refers to an identification number of a JNL group.

The various SEQ#s include, for example, a JNCB SEQ# which is a SEQ# ofthe newest prepared JNL.

The OPEN M*N valid flag refers to a flag set if there exist two or morePDKCs.

The freeze information refers to information indicating whether or not aJNL is in a freeze state.

The freeze state setting time refers to start time of the freeze state.

Hereinafter, a process performed in this embodiment will be described indetail.

FIG. 9 shows a process performed by a PDKC upon receiving a WR command.Hereinafter, this process will be described by way of an example of thePDKC 11P1.

For example, if the host 21B issues a WR command (S801), the PDKC 11P1receives the WR command (S802). The I/O processing unit 52P1 writes WRdata according to the WR command in the CM 33P1.

The I/O processing unit 52P1 specifies a JNL group to which a PVOLdesignated by the received WR command (a WR destination PVOL) belongs,from the pair management table 57P1, and acquires freeze information inthe JNL group management table 58P1 corresponding to the specified JNLgroup (S803).

If the freeze information acquired at S803 indicates a freeze state (YESin S804), the I/O processing unit 52P1 enters into a sleep state (S805).Specifically, a response to the WR command received at S802 cannot betransmitted to the host 21B.

If the freeze information acquired at S803 indicates no freeze state (NOin S804), the I/O processing unit 52P1 acquires a pair statecorresponding to the WR destination PVOL from the pair management table57P1 (S806).

If the pair state acquired at S806 is “PAIR” or “Copy” (YES in S807),the JNL preparing unit 53P1 prepares a JNL including a JNCB having JNLdata, which is a replica of the WR data, and P site receipt TS# (S808).The prepared JNL is stored in the JVOL 17PJ1. The I/O processing unit52P1 reports an end to the host 21B (S809). Information elementsincluded in the JNCB include, for example, update time, which is timewhen the JNCB is prepared, update position information related to awrite destination of the WR data, and SEQ#, in addition to the P sitereceipt TS#, as shown in FIG. 18.

If the pair state acquired at S806 is “Suspend” (YES in S810), the I/Oprocessing unit 52P1 sets a bit corresponding to a WR destination blockaccording to the WR command as a differential bit in a differential bitmap corresponding to a pair constituting a WR destination PVOL (S811).Thereafter, the above-described S809 is performed.

Each of bits constituting the differential bit map corresponds to eachof blocks constituting the PVOL. If the pair state is changed from“Suspend” to “Copy,” a JNL including JNL data, which are a replica ofdata stored in a block (block in the PVOL), is prepared for each blockcorresponding to the differential bit, and the prepared JNL is stored inthe JVOL 17PJ1.

Although not shown in FIG. 9, if NO in S804, the WR data are written inthe WR destination block (block in the WR destination PVOL) designatedby the WR command.

FIG. 10 shows a process performed by a SDKC upon transmitting an RD-JNLcommand.

The JNL-RD processing unit 61S1 starts at regular intervals (S901).

The JNL-RD processing unit 61S1 acquires a JNL group state by referringto the JNL management table 58S1 (S902).

If the JNL group state acquired at S902 is “Active” (YES in S903), theJNL-RD processing unit 61S1 acquires an OPEN M*N valid flag by referringto the JNL management table 58S1 referred to at S902 (S904).

If the flag acquired at S904 does not indicate “Valid” (NO in S905), theJNL-RD processing unit 61S1 prepares an RD-JNL command parameter (S909)and issues an RD-JNL command including the parameter (S910).

If the flag acquired at S904 indicates “Valid” (YES in S905), the JNL-RDprocessing unit 61S1 acquires a restore completion TS# and an arrivalcompletion TS# by referring to the JNL management table 58S1 referred toat S902 (S906 and S907). Then, the JNL-RD processing unit 61S1 preparesan RD-JNL command parameter including the acquired restore completionTS# and arrival completion TS# (S908) and issues an RD-JNL commandincluding the parameter (S910).

An example of information elements included in the RD-JNL commandparameter is shown in FIG. 19. The RD-JNL command parameter prepared atS908 includes, for example, an RD-JNL mode, a JNL group number in aPDKC, the restore completion TS# and the arrival completion TS#. TheRD-JNL mode is, for example, “Special read.” On the other hand, theRD-JNL command parameter prepared at S909 does not include the restorecompletion TS# and the arrival completion TS#. In this case, the RD-JNLmode is “Normal read,” “Retry read,” or “Purge command.” The JNL groupnumber in the PDKC is a counter JNL group number corresponding to a JNLgroup number registered with the JNL management table 58S1 referred toat S902 (a number specified from the pair management table 57S1).

FIG. 11 shows a portion of a process performed by a PDKC upon receivingan RD-JNL command, and FIG. 12 shows the remaining of the process.

As shown in FIG. 11, when the PDKC 11P1 receives the RD-JNL command fromthe SDKC 11S1 (S1001), the JNL-RD receipt processing unit 54P1 acquiresa command parameter from the RD-JNL command (S1002) and acquires anRD-JNL mode from the parameter (S1003).

If the RD-JNL mode acquired at S1003 is “Normal read” or “Retry read”(YES in S1004) and if there exists a non-transmitted JNL (YES in S1006),the JNL-RD receipt processing unit 54P1 acquires the non-transmitted JNLfrom the JVOL 17PJ1 (S1007), prepares a transmission parameter (S1008)and transmits the JNL acquired at S1007 as RD data to the SDKC 11S1,along with the transmission parameter (S1009). On the other hand, ifthere is no non-transmitted JNL (NO in S1006), the JNL-RD receiptprocessing unit 54P1 prepares a transmission parameter (S1010) andtransmits a NO-JNL (information indicating that there is no JNL) as RDdata to the SDKC 11S1, along with the transmission parameter (S1011).

If the RD-JNL mode acquired at S1003 is “Purge command” (NO in S1004 andNO in S1101 of FIG. 12) and if there exists a JNL of transmissioncompletion in the JVOL 17PJ1 (YES in S1008), the JNL-RD receiptprocessing unit 54P1 performs a purge process to delete the JNL from theJVOL 17PJ1 (S1109), as shown in FIG. 12.

If the RD-JNL mode acquired at S1003 is “Special read” (NO in S1004 andYES in S1101 of FIG. 12) and if there exists a non-transmitted JNL (YESin S1103), the JNL-RD receipt processing unit 54P1 acquires the JNL fromthe JVOL 17PJ1 (S1104), as shown in FIG. 12. In addition, the JNL-RDreceipt processing unit 54P1 acquires a restore permission TS# from theJNL management table 58P1 having a JNL group number in the parameteracquired at S1002 (S1105). The JNL-RD receipt processing unit 54P1prepares a transmission parameter including the acquired restorepermission TS# (S1106) and transmits the JNL acquired at S1104, as RDdata, to the SDKC 11S1, along with the transmission parameter (S1107).On the other hand, if there is no non-transmitted JNL (NO in S1103), theJNL-RD receipt processing unit 54P1 acquires a P site receipt TS# and arestore permission TS# from the JNL management table 58P1 having the JNLgroup number in the parameter acquired at S1002 (S1110 and S1111),prepares a transmission parameter including the acquired TS#s (S1112)and transmits a NO-JNL (information indicating that there is no JNL) asRD data to the SDKC 11S1, along with the transmission parameter (S1113).

FIG. 13 shows a process performed by a SDKC upon receiving RD data.

When the SDKC 11S1 receives RD data from the PDKC 11P1, the JNL-RDprocessing unit 61S1 starts (S1201).

The JNL-RD processing unit 61S1 checks an issued RD-JNL command for thereceipt of the RD data (S1202).

If the RD-JNL mode of the RD-JNL command checked at S1202 is “Normalread” or “Retry read” (YES in S1203) and if a JNL is included in the RDdata (YES in S1204), the JNL-RD processing unit 61S1 acquires the JNLfrom the RD data (S1205) and writes the acquired JNL in the JVOL 17SJ1(S1206).

If the RD-JNL mode of the RD-JNL command checked at S1202 is “Specialread” (NO in S1203 and YES in S1207) and if a JNL is included in the RDdata (YES in S1208), the JNL-RD processing unit 61S1 acquires a restorepermission TS# (hereinafter “X” in FIG. 13) from the RD data and setsthe acquired X as a restore permission TS# in the JNL management table58S1 corresponding to the RD-JNL command checked at S1202 (S1209). Inaddition, the JNL-RD processing unit 61S1 acquires a P site receipt TS#(hereinafter “W” in FIG. 13) from the RD data and sets the acquired W asan arrival completion TS# in the JNL management table 58S1 correspondingto the RD-JNL command checked at S1202 (S1210). In addition, the JNL-RDprocessing unit 61S1 acquires the JNL from the RD data (S1211) andwrites the acquired JNL in the JVOL 17SJ1 (S1212). On the other hand, ifno JNL is included in the RD data (NO in S1208), the JNL-RD processingunit 61S1 acquires a restore permission TS# (hereinafter “Y” in FIG. 13)from the RD data and sets the acquired Y as a restore permission TS# inthe JNL management table 58S1 corresponding to the RD-JNL commandchecked at S1202 (S1213). In addition, the JNL-RD processing unit 61S1acquires a P site receipt TS# (hereinafter “Z” in FIG. 13) from the RDdata and sets the acquired Z as an arrival completion TS# in the JNLmanagement table 58S1 corresponding to the RD-JNL command checked atS1202 (S1214).

In the meantime, the RM 23, which issues a TS# at regular intervals asdescribed above, is operated based on a definition file shown in FIG.14.

The definition file 69 includes, for example, the following information:

(14-1) Information on all PDKCs which become issuance sources of TS#(for example, DKC manufacture serial numbers)

(14-2) Command device number for each PDKC (device number designated atthe time of issuing TS#)

(14-3) Information on each PVOL of each PDKC (for example, volume number(LDEV#) of PVOL)

(14-4) TS issuance interval (time interval between issuance of TS# andnext issuance of TS#)

On the basis of the definition file 69 shown in FIG. 14, the RM 23issues a TS setting command to a DKC of Serial#64034 (hereinafter PDKC11P1) and a DKC of Serial#64045 (hereinafter PDKC 11P2) at a TS issuanceinterval designated by the file 69. As shown in FIG. 15, the TS settingcommand includes the following information elements (15-1) and (15-3):

(15-1) Command code (here, code meaning a TS setting command)

(15-2) JNL group number

(15-3) TS#

As shown in FIG. 16, if a PDKC receives the TS setting command (S1401),the TS processing unit 51P1 acquires a TS# from the command (S1402),sets the acquired TS# as a P site receipt TS# in the JNL managementtable 58P1 corresponding to a JNL group number in the command (S1403),and reports a process end to the RM 23 (S1404).

The RM 23 transmits a restore permission TS# to the PDKCs 11P1 and 11P2at regular intervals, for example.

Specifically, for example, as shown in FIG. 17, the RM 23 issues a stateacquisition command to the DKC of Serial# 64034 (PDKC 11P1) (S1301). Thestate acquisition command includes, for example, a JNL group number. Ifthe PDKC 11P1 (11P2) receives the state acquisition command as shown inFIG. 21 (S1901), for example, the TS processing unit in the 51P1acquires a micro version as shown in FIG. 21 (S1902). In addition, theTS processing unit 51P1 acquires a JNL group state, a P site receiptTS#, an arrival completion TS# and a restore completion TS# (S1903,S1904, S1905 and S1906) by referring to the JNL management table 58P1corresponding to the JNL group number in the state acquisition command,prepares response information (see FIG. 20) including the informationacquired at S1902 to S1906 (S1907), and transmits the responseinformation to the RM 23 (S1908).

As shown in FIG. 17, upon receiving the response information on thestate acquisition command (S1302), the RM 23 checks the micro versionincluded in the response information (S1303).

If it is specified that when the PDKC 11P1 supports an OPEN M*Naccording to S1303 (or other methods) (YES in S1304), the RM 23 acquiresa JNL group state from the response information received at S1302(S1305). If the JNL group state is “Active,” the RM 23 sets the arrivalcompletion TS# included in the response information, as a TS# of thePDKC 11P1 (S1307) The same processes as the above-described S1301 toS1307 are performed for the PDKC 11P2 (S1308 to S1314).

The RM 23 determines the lowest value of the TS# set at S1307 and theTS# set at S1314 as a restore permission TS# to be issued (S1315). TheRM 23 transmits the determined restore permission TS# to the PDKCs 11P1and 11P2 (S1316 and S1317) The restore permission TS# is transmittedalong with the JNL group number included in the state acquisitioncommand issued at S1301 and S1308, for example.

The PDKC 11P1 (11P2) registers the restore permission TS# received fromthe RM 23 with the JNL group management table 58P1 corresponding to theTS#. In addition, the PDKC 11P1 (11P2) transmits the restore permissionTS# registered with the JNL group management table 58P1 to the SDKC 11S1(11S2). The restore permission TS# may be either actively transmitted bythe PDKC 11P1 (11P2) or for example transmitted at the time oftransmission of RD data according to the RD-JNL command from the SDKC11S1 (11S2). The SDKC 11S1 (11S2) registers the restore permission TS#received from the PDKC 11P1 (11P2) with the JNL group management table58S1 corresponding to the TS#.

As described above, in the SDKC 11S1 (11S2), up to a JNL having a TS#smaller by one than the restore permission TS# is reflected in the SVOL17S1 (17S2). The reason for this is as shown in FIG. 22. That is,assuming that an arrival completion TS# in the SDKC 11S1 is “5” and anarrival completion TS# in the SDKC 11S2 is “4,” although the lowestvalue of a restore permission TS# becomes “4,” the SDKC 11S2 may receivea JNL including a TS# 4 in the future. That is, JNL data for the TS# 4are not determined in the SDKC 11S2. On this account, up to a JNLincluding a TS# 3 smaller by one than the restore permission TS# 4 isreflected in the SDKCs 11S1 and 11S2.

According to the above description, if there is no non-transmitted JNL,upon receiving the RD-JNL command, the PDKC 11P1 (11P2) transmits theNO-JNL including the P site receipt TS#. Thereby, although a PDKC whichdoes not receive a WR command for a moment is included in the DKC group18, the number of non-reflected JNLs excessively increases in a SDKCpaired with a PDKC other than the PDKC. Accordingly, it can be expectedto prevent a JVOL from being fully filled with JNLs. A detailed examplefor this is as shown in FIGS. 23A and 23B.

As shown in FIG. 23A, for example, under a situation where the PDKC 11P2does not receive a WR command for a moment, if the PDKC 11P2 receives anRD-JNL command and thus there is no non-transmitted JNL and if there isno function to transmit a NO-JNL, a P site receipt TS# 70 managed by thePDKC 11P2 is not transmitted to the SDKC 11S2 at all. Accordingly, anarrival completion TS# 5 managed by the SDKC 11S2 is not updated. As aresult, the restore permission TS# also remains at the same value as thearrival completion TS# 5 in the SDKC 11S2. Accordingly, although up to aJNL having the TS# 4 smaller by one than the restore permission TS# 5 isreflected in the SDKC 11S1, JNLs subsequent to a JNL having the next TS#5 are left in the JVOL 17SJ1 without being reflected. In addition, everytime the PDKC 11P1 receives a WR command, a new JNL is prepared. The newJNL is transmitted to the SDKC 11S1 and written in the JVOL 17SJ1. As aresult, the JVOL 17SJ1 is fully filled with JNLs, which may result inimpossibility of storage of more new JNLs.

To avoid such impossibility, in this embodiment, as shown in FIG. 23B(that is, as described previously), for example, under a situation wherethe PDKC 11P2 does not receive the WR command for a moment, if the PDKC11P2 receives the RD-JNL command and thus there is no non-transmittedJNL, the PDKC 11P2 transmits the RD data, which has no JNL but has the Psite receipt TS#, that is, the NO-JNL, to the SDKC 11S2. Thereby,according to the example shown in FIG. 23B, the arrival completion TS#sacquired from the SDKCs 11S1 and 11S2 all become “70” and thus therestore permission TS# also becomes “70”, so that the SDKC 11S1 canreflect up to a JNL having a TS# 69 in the SVOL 17S1. Accordingly, itcan be expected to prevent the JVOL 17SJ1 from being fully filled withnon-reflected JNLs.

In addition, according to the above description, the RM 23 transmits afreeze command to the PDKCs 11P1 and 11P2 before transmitting the TSsetting command. Thereby, an update order can be maintained.

For example, as shown in FIG. 24, the RM 23 transmits the freeze commandto the PDKCs 11P1 and 11P2 (S241A and S241B), and then transmits a TSsetting command including a TS# 5 to the PDKCs 11P1 and 11P2 (S242A andS242B).

The I/O issuing unit 25 starts independent of the RM 23. In other words,a timing at which a TS setting command is issued does not depend on atiming at which a WR command is issued. Thus, as shown in FIG. 24,between when the RM 23 transmits the TS setting command (TS# 5) to thePDKC 11P1 and when the RM 23 transmits the TS setting command (TS# 5) tothe PDKC 11P2 (that is, between S242A and S242B), the I/O issuing unit25 may transmit one WR command to the PDKC 11P1 while transmittinganother WR command to the PDKC 11P2. However, at this point, since thePDKCs 11P1 and 11P2 are all in the freeze state, the PDKCs 11P1 and 11P2respond to the received WR commands after the freeze state is released.During the freeze state, the TS# 5 in the TS setting command issued inS241A and S241B are managed as a P site receipt TS# in the PDKCs 11P1and 11P2. Accordingly, if the freeze state is released, when a WRprocess is performed in response to the WR command in S242A, a JNL 2401including the TS# 5 is prepared in the PDKC 11P1, as shown in FIG. 24.Likewise, when a WR process is performed in response to the WR commandin S242B, a JNL 2402 including the TS# 5 is prepared in the PDKC 11P1.

From the above description, if a freeze command is not issued before theTS setting command is received, the following problem may occur. Thatis, as shown in FIG. 25, when the PDKC 11P1 and 11P2 receive the WRcommand between the above S242A and S242B, the PDKC 11P1 prepares a JNL2501 including the TS# 5 in the TS setting command received in S242Awhile the PDKC 11P2 prepares a JNL 2502 including the P site receipt TS#4 before the TS setting command is received in S242B. As a result, forall that the PDKC 11P1 receives the WR command earlier, the TS# includedin the JNL prepared in the PDKC 11P2 becomes a newer value. That is, anupdate order becomes reversed.

Accordingly, as described above, as the RM 23 transmits the freezecommand before transmitting the TS setting command, the update order inthe DKC group 18 can be maintained.

FIG. 26 shows a process performed by a PDKC upon receiving a freezecommand.

For example, when the PDKC 11P1 receives a freeze command (S2601), theF/R receipt processing unit 55P1 acquires freeze information from theJNL group management table 58P1 (hereinafter referred to as “object JNLgroup management table” in the description about FIG. 26) correspondingto a JNL group number designated by the freeze command (S2602).

If the freeze information acquired at S2602 does not indicate a freezestate (NO in S2603), the F/R receipt processing unit 55P1 acquirescurrent time from, for example, a timer (S2606) and writes the acquiredcurrent time as freeze state setting time in the object JNL groupmanagement table (S2607). In addition, the F/R receipt processing unit55P1 sets the freeze information indicating present status of freezingin the object JNL group management table (S2608).

On the other hand, if the freeze information acquired at S2602 indicatesthe freeze state (YES in S2603), the F/R receipt processing unit 55P1acquires freeze state setting time from the object JNL group managementtable (S2604). If a difference between time indicated by the freezestate setting time acquired at S2604 and the current time does notexceed predetermined time (that is, overtime does not occur) (NO inS2605), the present process is ended. On the other hand, if overtimeoccurs (YES in S2605), the F/R receipt processing unit 55P1 outputs alog for analysis (S2609) and updates the freeze information registeredwith the object JNL group management table to information indicating nofreeze state (S2610).

Although it is shown in FIG. 26 that the freeze state is released by theovertime, instead of or in addition to this, the freeze state may bereleased by a command from the RM 23. Hereinafter, the release commandfor the freeze state is referred to as “RUN command.”

FIG. 27 shows a process performed by a PDKC upon receiving a RUNcommand.

For example, when the PDKC 11P1 receives a RUN command, the F/R receiptprocessing unit 55P1 acquires freeze information from the JNL groupmanagement table 58P1 corresponding to a JNL group number designated bythe RUN command (hereinafter referred to as “object JNL group managementtable” in the description about FIG. 27) (S2702).

If the freeze information acquired at S2602 indicates a freeze state(YES in S2703), the F/R receipt processing unit 55P1 updates the freezeinformation registered with the object JNL group management table toinformation indicating no freeze state (S2704).

In the meantime, in this embodiment, if a disturbance occurs in the RM23, as shown in FIG. 28, and thereafter if the RM 23 is restored out ofthe disturbance, the RM 23 determines what TS# is to be set in a TSsetting command to be first transmitted after the restoration. In moredetail, a process shown in FIG. 29 is performed.

When the RM 23 transmits a state acquisition command for any JNL group(hereinafter referred to as “object group” in the description about FIG.29) to the PDKC 11P1 (S2901), the RM 23 receives a P site receipt TS#corresponding to the object JNL group (TS# 100 in the example of FIG.28), which is currently stored in the PDKC 11P1, from the PDKC 11P1(S2902). Likewise, when the RM 23 transmits a state acquisition commandfor the object JNL group to the PDKC 11P2 (S2903), the RM 23 receives aP site receipt TS# corresponding to the object JNL group (TS# 99 in theexample of FIG. 28), which is currently stored in the PDKC 11P2, fromthe PDKC 11P1 (S2904). The RM 23 judges the highest value of the P sitereceipt TS#s received at S2902 and S2904 (S2905) and determines a value(i.e., “101”) larger by one than the judged highest value (“100” in theexample of FIG. 28) as a TS# to be issued (S2906). The RM 23 transmits aTS setting command including the determined TS# to the PDKCs 11P1 and11P2 (S2907).

In the meantime, in this embodiment, if there occurs a disturbancebetween the host 21A and one PDKC (for example, the PDKC 11P1) (S3001),as shown in FIG. 30, a process shown in FIGS. 30 to 32 is performed. Inthe following description, on the basis of FIG. 5, pair state modifyingunits of the PDKC 11P2 and the SDKC 11S2 are referred to as pair statemodifying units 56P2 and 56S2, respectively, and pair management tablesof the PDKC 11P2 and the SDKC 11S2 are referred to as pair managementtables 57P2 and 57S2, respectively.

For example, if the RM 23 receives no response from the PDKC 11P1although the RM 23 issues a state setting command to the PDKC 11P1, theRM 23 detects disability of communication with the PDKC 11P1 (S3002).Then, the RM 23 instructs a communicable PDKC 11P2 to be suspended(S3003). In response to this instruction, the pair state modifying unit56P2 in the PDKC 11P2 updates a pair state of all pairs to “Suspend” byreferring to the pair management table 57P2 (S3004A). When it isdetected that the pair state of all pairs in the PDKC 11P2 is updated to“Suspend,” the pair state modifying unit 56S2 in the SDKC 11S2 updatesall pair states in the pair management table 57S2 to “Suspend” (S3004B).The update of the pair state of all pairs in the PDKC 11P2 to “Suspend”may be either actively reported from the PDKC 11P2 to the SDKC 11S2 orreported with a response to an inquiry from the SDKC 11S2.

Thereafter, if the RM 23 is restored out of the disturbance occurring inS3001 (S3101), as shown in FIG. 31, the RM 23 makes a state between thePDKC 11P1 related to the restoration out of the disturbance and the SDKC11S1 equal to a state between the PDKC 11P2 and the SDKC 11S2. That is,the RM 23 detects from the PDKC 11P2 that all pair states in the PDKC11P2 are set to “Suspend” (S3102). In this case, the RM 23 instructs thePDKC 11P1 to be suspended (S3104). In response to this instruction, thepair state modifying unit 56P1 in the PDKC 11P1 updates a pair state ofall pairs to “Suspend” by referring to the pair management table 57P1(S3105A). When it is detected that the pair state of all pairs in thePDKC 11P1 is updated to “Suspend,” the pair state modifying unit 56S1 inthe SDKC 11S1 updates all pair states in the pair management table 57S1to “Suspend” (S3105B).

Thereafter, the RM 23 instructs the PDKC 11P1 to be resynchronized(S3201A). In response to this instruction, the pair state modifying unit56P1 in the PDKC 11P1 updates a pair state of all pairs from “Suspend”to “PAIR” by referring to the pair management table 57P1 (S3202A). Whenit is detected that the pair state of all pairs in the PDKC 11P1 isupdated to “PAIR,” the pair state modifying unit 56S1 in the SDKC 11S1updates all pair states in the pair management table 57S1 from “Suspend”to “PAIR” (S3202B). Also, the pair state transits from “Suspend,”through “Copy,” to “PAIR.” During the pair state “Copy,” a differencebetween a PVOL and a SVOL is reflected in the SVOL, thereby making thecontents of the PVOL equal to the contents of the SVOL and thus settingthe pair state to be “PAIR.”

The above-described processes are performed in the same way for the PDKC11P2 and the SDKC 11S2 (S3201B, S3202C and S3202D).

Although the present invention has been shown and described by way ofthe exemplary embodiments, the present invention is not limited to theexemplary embodiments but it is to be understood that the presentinvention may be modified and changed in various ways without departingfrom the spirit and scope of the invention. For example, the presentinvention can be applied to computer systems having mainframe computerswithout being limited to computer systems having open systems.

1. A computer system comprising: one or more primary storage systems;one or more secondary storage systems; and one or more computers,wherein one of the one or more computers includes a storage managingunit and at least the remaining of the one or more computers includes anI/O issuing unit, wherein the storage managing unit issues an ID next toan immediately previously issued ID to the one or more primary storagesystem at regular or irregular intervals, wherein the I/O issuing unitissues a write command designating a primary volume, wherein each of theone or more primary storage systems has a primary volume and a primaryjournal memory region, writes write data in the primary volume inresponse to the write command, prepares a journal including journaldata, which is a replica of the write data, and the newest ID from thestorage managing unit, stores the prepared journal in the primaryjournal memory region, and transmits the journal to a secondary storagesystem of a connection site, wherein the secondary storage system has asecondary volume and a secondary journal memory region, writes areceived journal in the secondary journal memory region, and reflects upto a particular journal of one or more non-reflected journals stored inthe secondary journal memory region in the secondary volume, wherein theparticular journal is a journal having an ID older by one than theoldest ID of the newest arrival completion IDs in the one or moresecondary storage systems, and wherein journal data in the journal iswritten in the secondary volume in the reflection of the journal in thesecondary volume.
 2. The computer system according to claim 1, whereineach of two or more primary storage systems is connected to at least oneof the one or more secondary storage systems and has the following (P1)to (P7): (P1) a primary ID memory region that stores the following(P1-1): (P1-1) the primary newest ID which is the newest ID receivedfrom the storage managing unit, (P2) the primary volume which is alogical volume designated by the write command from the I/O issuingunit, (P3) an I/O processing unit that writes write data according tothe write command in the primary volume, (P4) a journal preparing unitthat prepares a journal including the following (P4-1) and (P4-2):(P4-1) journal data which is a replica of the write data, and (P4-2)journal management information including update position informationrelated to a position of a write site of the write data and the primarynewest ID stored in the primary ID memory region, (P5) the primaryjournal memory region that is a memory region which stores the preparedjournal, (P6) a journal transmitting unit that transmits a journalstored in the primary journal memory region to the secondary storagesystem of the connection site, the secondary storage system having asecondary volume paired with the primary volume indicated by the updateposition information in the journal, and (P7) an ID processing unit thatwrites an ID received from the storage managing unit in the primary IDmemory region, as the primary newest ID, wherein the secondary storagesystem has the following (S1) to (S5): (S1) a secondary ID memory regionthat stores the following (S1-1): (S1-1) an arrival completion ID whichis the primary newest ID included in journal management information in ajournal received from a primary storage system of a connection site,(S2) the secondary journal memory region that stores the journalreceived from the primary storage system of the connection site, (S3)the secondary volume that is a logical volume which becomes a write siteof journal data in a journal stored in the secondary journal memoryregion, (S4) a restore processing unit that reflects up to a particularjournal of one or more non-reflected journals stored in the secondaryjournal memory region in the secondary volume paired with a primaryvolume of a write site of write data corresponding to journal data in ajournal to be reflected, and (S5) an ID notifying unit that notifies anarrival completion ID stored in the secondary ID memory region, whereinthe storage managing unit notifies a restore permission ID which is theoldest ID of arrival completion IDs in the one or more secondary storagesystems or an ID older by one than the oldest ID, and wherein theparticular journal is a journal having an ID older by one than therestore permission ID or the restore permission ID.
 3. The computersystem according to claim 2, wherein the (S5) notifies the primarystorage system of the connection site of the arrival completion ID,wherein the (P1) further stores the following (P1-2): (P1-2) the arrivalcompletion ID received from the secondary storage system of theconnection site, wherein the storage managing unit transmits an IDinquiry to the two or more primary storage systems, receives an arrivalcompletion ID from the two or more primary storage systems in responseto the ID inquiry, and notifies each primary storage system of theoldest ID of received two or more arrival completion IDs or an ID olderby one than the oldest ID, as the restore permission ID, wherein the(P7) notifies the secondary storage system of the connection site of therestore permission ID received from the storage managing unit, andwherein the (S1) further stores the following (S1-2): (S1-2) the restorepermission ID received from the primary storage system of the connectionsite.
 4. The computer system according to claim 3, wherein the (S5)transmits a journal read command including an arrival completion ID tothe primary storage system of the connection site, and wherein, uponreceiving the journal read command, the (P6) writes the arrivalcompletion ID included in the journal read command in the primary IDmemory region and transmits a non-transmitted journal of one or morejournals stored in the primary journal memory region to a secondarystorage system of a transmission source of the journal read command. 5.The computer system according to claim 4, wherein, if there exists nonon-transmitted journal in the primary journal memory region, the (P6)transmits information including the primary newest ID stored in theprimary ID memory region to the secondary storage system of thetransmission source of the journal read command.
 6. The computer systemaccording to claim 5, wherein the storage managing unit transmits afreeze command, which is a command reserving an I/O, to a primarystorage system of a transmission destination of the ID inquiry beforetransmitting the ID inquiry, and transmits the ID inquiry aftertransmitting the freeze command, and wherein, upon receiving the freezecommand, the (P3) enters into a freeze state in which a write process towrite the write data according to the write command from the I/O issuingunit in the primary volume designated by the write command is notperformed, and performs the write process after the freeze state isreleased.
 7. The computer system according to claim 6, wherein thestorage managing unit transmits the freeze command before transmittingthe ID and transmits the ID after transmitting the freeze command. 8.The computer system according to claim 1, wherein the secondary storagesystem notifies the primary storage system of the connection site of thearrival completion ID, wherein the storage managing unit transmits an IDinquiry to each primary storage system, receives an arrival completionID from the each primary storage system in response to the ID inquiry,and notifies the each primary storage system of the oldest ID ofreceived arrival completion IDs or an ID older by one than the oldestID, as the restore permission ID, and wherein the particular journal isa journal having an ID older by one than the restore permission ID orthe restore permission ID.
 9. The computer system according to claim 8,wherein the secondary storage system transmits a journal read command tothe primary storage system of the connection site, wherein, uponreceiving the journal read command, each primary storage systemtransmits a non-transmitted journal of one or more journals stored inthe primary journal memory region to the secondary storage system of theconnection site, wherein an arrival completion ID is included in thejournal read command, and wherein an arrival completion ID that thestorage managing unit receives from each primary storage system is thearrival completion ID included in the journal read command.
 10. Thecomputer system according to claim 1, wherein the secondary storagesystem transmits a journal read command to the primary storage system ofthe connection site, wherein, upon receiving the journal read command,each primary storage system transmits a non-transmitted journal of oneor more journals stored in the primary journal memory region to thesecondary storage system of the connection site, and, if there exists nonon-transmitted journal, transmits information, which includes thenewest ID from the storage managing unit, to the secondary storagesystem of the connection site.
 11. The computer system according toclaim 1, wherein the storage managing unit transmits a freeze command,which is a command reserving an I/O, to a primary storage system of atransmission destination of the ID inquiry before transmitting the IDinquiry, and transmits the ID inquiry after transmitting the freezecommand, and wherein, upon receiving the freeze command, each primarystorage system enters into a freeze state in which a write process towrite the write data according to the write command from the I/O issuingunit in the primary volume designated by the write command is notperformed, and performs the write process after the freeze state isreleased.
 12. The computer system according to claim 1, wherein thestorage managing unit transmits a freeze command, which is a commandreserving an I/O, to a primary storage system of a transmissiondestination of the ID inquiry before transmitting the ID inquiry, andtransmits the ID after transmitting the freeze command, and wherein,upon receiving the freeze command, each primary storage system entersinto a freeze state in which a write process to write the write dataaccording to the write command from the I/O issuing unit in the primaryvolume designated by the write command is not performed, and performsthe write process after the freeze state is released.
 13. The computersystem according to claim 1, wherein, upon detecting that at least oneof two or more primary storage systems is restored out of a disturbance,the storage managing unit transmits an ID inquiry to the two or moreprimary storage systems, receives the primary newest ID from the two ormore primary storage systems, and sets an ID newer by one than thenewest ID of received two or more primary newest IDs as an ID to befirst transmitted after the restoration out of the disturbance.
 14. Thecomputer system according to claim 1, wherein, upon detecting that atleast one of two or more primary storage systems has a disturbance, thestorage managing unit instructs a primary storage system detected tohave no disturbance to set a state of a pair including a primary volumeof the primary storage system as a copy suspend state, and thereafter,upon detecting restoration out of the disturbance, instructs a primarystorage system related to the restoration out of the disturbance to seta state of a pair including a primary volume of the primary storagesystem as a copy suspend state, and thereafter, instructs the two ormore primary storage systems to make states of pairs including primaryvolumes equal to each other, and wherein, in response to the instructionto make the states of the pairs equal to each other, a differencebetween the primary volume and the secondary volume is reflected in thesecondary volume so that the state of the pair becomes the state inwhich the contents of the primary volume becomes equal to the contentsof the secondary volume.
 15. A method of controlling a remote copyimplemented by a computer system including one or more primary storagesystems, one or more secondary storage systems, and one or morecomputers, wherein a particular computer of the one or more computersissues an ID next to an immediately previously issued ID to each of theone or more primary storage systems at regular or irregular intervals,wherein, upon receiving a write command from one of the one or morecomputers, each primary storage system writes write data in a primaryvolume, prepares a journal including journal data, which is a replica ofthe write data, and the newest ID from the particular computer, storesthe prepared journal in a primary journal memory region, and transmitsthe journal to a secondary storage system of a connection site, whereinthe secondary storage system writes a received journal in a secondaryjournal memory region, and writes journal data in journal up to aparticular journal of one or more journals stored in the secondaryjournal memory region in a secondary volume, and wherein the particularjournal is a journal having an ID older by one than the oldest ID of thenewest arrival completion IDs in the one or more secondary storagesystems.